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ABSTRACT 

An approach to incorporating artificial neural net- 
works in nonlinear, adaptive control systems is de- 
scribed. The controller contains three principal ele- 
ments: a nonlinear-inverse-dynamic control law 
whose coefficients depend on a comprehensive 
model of the plant, a neural network that models 
system dynamics, and a state estimator whose out- 
puts drive the control law and train the neural 
network. Attention is focused on the system-iden- 
tification task, which combines an extended 
Kalman filter with generalized-spline function ap- 
proximation (e g., basis splines, back-propagation 
feedforward networks, or cerebellar model articula- 
tion controller networks). Continual learning is 
possible during normal operation (without taking 
the system off line for specialized training). Non- 
linear-inverse-dynamic control requires smooth 
derivatives as well as function estimates, imposing 
stringent goals on the approximating technique. 

INTRODUCTION 

Current research on artificial neural networks has 
been spurred by the need for pattern recognition, 
optimization, and control algorithms that can learn 
from experience. The biological paradigm provides 
an attractive starting point, as learning is the crux of 
living activity. Nevertheless, it remains to be seen 
whether or not biologically-motivated algorithms 
(i.e., artificial neural networks) are computationally 
appealing, particularly as compared to more con- 
ventional numerical techniques for function ap- 
proximation. These techniques also can "learn" or 
"be taught," although those terms rarely are used 
for the process of fitting curves or determining coef- 
ficients from empirical data. 

The most important question appears to be how 
well neural networks generalize multivariate func- 
tions of high input dimension. Basis splines (or B- 
splines) represent an efficient, modern, conven- 
tional technique (1, 2). A univariate B-spline func- 
tion approximation can be trained quickly and accu- 
rately using deterministic or averaging techniques, 
and the resulting spline approximant has nice nu- 
merical properties, such as positivity and compact 
support. Multivariate B-splines can be computed, 
but they may become cumbersome in both training 
and operation if the number of independent vari- 
ables is much larger than two. Furthermore, the 
multivariate B-spline requires a fairly rigid defini- 
tion of the input space, resulting in potentially high 
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computation times and storage requirements. 
Neural networks can, in principle, gain efficiency 
through automatically identifying and concentrat- 
ing on significant receptive regions in the input 
space [3,4]. 

Neural networks can be incorporated in nonlinear 
control logic in at least two ways. The first is as a 
generalized feedforward control function that maps the 
space of desired command responses into corre- 
sponding control settings. This neural network is 
conceptually straightforward, as it is based on the 
inverse of the plant's response to control inputs [5]. 
The second is to use the neural network as a model 
of plant dynamics , which is embodied in the control 
law but is distinct from the command /control spec- 
ification [as in nonlinear-inverse-dynamic (NID) 
control]. The first approach may lead to more com- 
pact implementation, while the second may pro- 
vide more flexibility for multi-modal or fault-toler- 
ant control. 

In the remainder of the paper, we discuss elements 
of the second approach. After briefly stating the 
nonlinear control problem and its NID solution, a 
learning structure for neural networks based on an 
extended Kalman filter is described, and details of 
two network formulations are presented. It is 
shown that smooth neural network outputs are de- 
sirable and, in fact, required for NID control. The 
back-propagation feedforward network has suffi- 
cient smoothness as normally defined, but the cere- 
bellar model articulation controller network must 
be modified to eliminate quantization effects. 

THE NONLINEAR CONTROL PROBLEM 

Consider the control of a nonlinear dynamic plant 
as described by, 

X = f(x) + g(x, u) + w (1) 

y = h(x) (2) 

z - h 2 (x) + n (3) 

with x and w € R n , u, y e R m , and z and w e R 1 . f 
and g are smooth vector fields in R n ; h and h 2 are 
smooth vectors fields in R m and R 1 . x represents 
the dynamic state, u is its control, w is the distur- 
bance input, y is the response vector, z is the mea- 
surement of the process, and n is the measurement 
error. 

For illustration, assume that eq. 1 and 2 can be writ- 
ten as, 

x = f(x) + G(x) u + w (4) 



y = Hx ( 5 ) 

with w = 0 and z = x. Define the derivative response 
vector by termwise differentiation of eq. 5 and 
substitution of eq. 4, repeated until the control af- 
fects each element of yW linearly [6, 7J: 

y<d> = f*(x) 4 G*(x) u (6) 

G*(x) is guaranteed to be nonsingular, so it is possi- 
ble to construct an integrator-decoupling control /atr of 
the form, 

u = c(x) 4 C(x)v (7) 

where, 

C(x) - [G*(x)]-1 (8) 

c(x) = -[G*(x)H Hx) (9) 

and v is a command input vector. Together with 
additional compensation to provide satisfactory 
closed-loop stability and command response, eq. 7 
forms a nonlinear-inverse-dynamic control law. 

For typical response vector specifications in aircraft 
control applications [8,9], one to three differentia- 
tions are required to bring out the control effect in 
y< d >; hence, up to two partial derivatives of f(x) and 
G(x) (with respect to x) must exist in those cases. 
Our goal is to approximate an uncertain but observ- 
able portion of the system model using neural net- 
works. Consequently, neural network outputs 
must be smooth enough for the differentiation and 
inversion contained in eqs. 6, 8, and 9 to take place. 

THE SYSTEM IDENTIFICATION PROBLEM 

Consider a system described by eq. 1-3 with w = 0 
and z = x, in which f(x) is known without error and 
g(x,u) is subject to uncertainty or change. Let g(t) 
represent the value of g(x,u) at time t during some 
period of system operation; then g(t) could be esti- 
mated precisely as 

g(t) = x(t) - f[x(t)] (10) 

and (g(t), x(t), u ( t) } would form a suitable training 
set for a functional approximation to g(x,u). During 
the course of normal system operation, it is as- 
sumed that x(t) and u(t) would provide a succession 
of training sets that were sufficiently rich for func- 
tion approximation, and it would be possible to 
"replay" system trajectories off line for improved 
learning. There is no restriction on the form of 
g(x,u); it is suggested here that g(x,u) be represented 
by one or more neural networks. 

More realistically, w and n are not zero, and z is not 
x, so eq. 10 cannot be evaluated. However, with the 
usual observability requirements, it is possible to 
make estimates of g(t) and x(t) using an Extended 
Kalman Filter, as described, for example, in [10]. 
Following a procedure analogous to Estimation 
Before Modeling [11-15], the state vector is augmented 
to include g(t) and enough derivatives of g(t) to as- 
sure a smooth estimate of g(t). [In the off-line air- 


craft system identification applications presented in 
[11-15] (Fig. 1), g(t) was treated as an integrated ran- 
dom-walk (Gauss-Markov) process, necessitating 
the estimation of g(t) and g(t) as well as g(t). 
Improved estimates were obtained by following the 
filtering step with a reverse-time modified Bryson- 
Frazier (MBF) smoothing step [16].J[ In the current 
application, the filtered estimates g(t) and x(t^ are 
used with u(t) to bain the neural network g(x, u). 
Smoothed estimates could be used for further off- 
line training improvement. [See [17] for a related 
application of filtering and smoothing]. 


z(t) u(t) 



Figure 1. Basic Estimation Before Modeling. 


For the airplane example, x(t) is an 11 -component 
vector representing 3 components (each) of transla- 
tional and rotational rates, 4 quaternion compo- 
nents, and altitude. f[x(t)] represents kinematic and 
inertial effects, w ? hile the vector g(t) has just 6 non- 
zero terms due to aerodynamic, thrust, and control 
effects. These terms are axial, side, and normal spe- 
cific forces (X(t), Y(t), and Z(t)] and roll, pitch, and 
yaw* specific moments [L(t), M(t), and N(t)]. Conse- 
quently, 6 parallel neural networks would be 
formed for this application. Aerodynamic effects 
can be normalized using air density, airspeed, refer- 
ence area, and reference length; hence, the neural 
networks would represent the non-dimensional 
coefficients Cx(x,u), Cy(x,u), Cz(x,u), Cj(x,u), 
C m (x,u), and C n (x,u). 

The neural network can be integrated with N1D 
control, as shown in Fig. 2. The dashed arrow from 
the estimation model block to the nonlinear con- 
trol law block represents an update of the func- 
tional form of the control law rather than the direct 
feedback path of the other connections. 



Figure 2. Integration of Neural-Network System 
Identification writh Nonlinear-Inverse-Dynamic 
Control. 
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NEURAL NETWORK ESTIMATION SCHEMES 

Back-propagation feedforward neural networks and 
CMAC neural networks are particularly suitable for 
system identification. Each represents a different 
structural model of a process and presents unique 
capabilities for the system identification process. 
Although neural network techniques appear very 
different from classic techniques such as B-splines, 
under close examination there are strong similari- 
ties As discussed in [18k most neural networks, in- 
cluding the two models discussed here, can be 
viewed as generalized splines. This observation 
opens the door on a large body of knowledge about 
function approximation. A brief description of the 
two networks follows. Further details about each 
can be found in [19k 

Back-Propagation Feedforward Network 

One of the simplest, and most used, artificial neural 
network is the back-propagation feedforward net- 
work [20k A feedforward neural network is con- 
structed from a weighted interconnection of simple 
nonlinear elements called nodes. The nodes are 
separated into layers , with the input to each node 
exclusively from the previous layer and the output 
connected only to the following layer. The orderly 
construction leads to a simple recursive definition 

t (k) e w (k-l) x (k-l) (n) 

x (k) = s (k>[, ( k)] (12) 

for layers k = 1 to N. The activation function vector, 
is formed by concatenating the scalar activa- 
tion functions contained in each node of a single 
layer: 

8 ,k) {t) = loi(tl) ... o n (t n )l T (13) 


algorithm. The choice of activation functions and 
numbers of layers and nodes is the subject of much 
research [e g. [21]]. 

Cerebellar Model Articulation Controller 

The Cerebellar Model Articulation Controller 
(CMAC) neural network [3, 4) is a generalization of 
a simple table look-up of a multi-input/single-out- 
put nonlinear function. Originally modeled on the 
uniform structure of the cerebellum, the basic 
CMAC divides the multi-dimensional input space 
into a series of overlapping hypercubes. A weight is 
assigned to each hypercube. The output of the 
CMAC for a specific input point is simply the sum 
of the weights of those hypercubes enclosing the 
input point. Applications to date have been in the 
areas of function approximation [3, 4, 17], robotics 
[5,22,23], and adaptive control [24]. 

Introducing an association space , A, the operation of 
the CMAC is best described by two mappings. The 
first goes from the input space, X , to the association 
space 

Q.: X-* A - (0, 1) N * (14) 

The elements of the binary selector vector , a, in the 
Na dimensional A are fixed by 

_ fl XG Aj 

a i~ l0 otherwise * 

where Aj is the j th receptive region. The receptive 
regions are the overlapping hypercubes that cover 
the input space. By overlapping the receptive re- 
gions such that any input lies within exactly C re- 
ceptive regions, a has C non-zero values. The size 
of C controls the generalization capability of the 
CMAC. 


One input to each layer is fixed at unity and serves 
as a threshold, as illustrated by Fig. 3. 



Figure 3. A Simple Feedforward Neural Network. 

Given the form of the activation function (usually 
the same in each node), the number of layers, and 
the number of nodes in each layer, the back-propa- 
gation rule [20] can be used to update the values of 
the weights, WM. The back-propagation rule at- 
tempts to minimize the mean-square error of the 
difference between the network output and a de- 
sired output value using a gradient-based descent 


The second mapping from a to the scalar output, y, 

A -» 7 (16) 

is simply a summation of the weights associated 
with the selected receptive regions. Defining a 
weight vector, w, of dimension Na, the inner prod- 
uct of w with a generates the desired output 

y = w T a (17) 

The basic CMAC (Fig. 4) quantizes the inputs based 
on the size of the overlap of the receptive regions. 
Since there is only one further linear operation per- 
formed, the output of the CMAC is a piecewise-con- 
tinuous function. 

Since the weights and output are connected by a 
simple linear operation, a learning algorithm is 
easy to prescribe. Each weight contributing to a par- 
ticular output value is adjusted by a fraction of the 
difference between the network output and the de- 
sired output. The fraction added is determined by 
both the speed of desired learning and the number 
of weights contributing to the output. 
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By its construction, the fidelity of the learned 
CMAC response is directly related to the quantiza- 
tion of the input space into receptive regions. For 
multi-dimensional input spaces, the number of re- 
ceptive regions, and therefore the number of 
weights, scales as 


= Q n ) (18) 

where Q is the number of number of divisions in 
each input dimension and n is the number of input 
dimensions. For even reasonably sized, multi-in- 
put problems, the number of weights quickly over- 
whelms the memory capacity of most computers. 
Fortunately, in many applications, such as control, 
only small regions of (or trajectories through) the 
input space are used. By implementing a many-to- 
few mapping of the weight vector using a random 
hash-coding technique [25], the number of necessary 
weights can be compressed to a reasonable size. For 
example, in [23], a CMAC was applied to a robotics 
problem with 15 inputs. The weight vector was 
compressed using a random hash function from 
the original large length down to lengths between 
32,768 and 8 weights. Virtually no degradation in 
performance was visible for lengths down to 1024. 

DIFFERENTIABILITY OF THE ESTIMATION 
SCHEMES 

The application to NID control imposes strong re- 
strictions on the estimation scheme to be used, in- 
cluding the the existence of smooth derivatives. 
The two neural network techniques described have 
differing capabilities in this regard. 

Recognizing that the overall function of the feed- 
forward network is 


x (N) = g[x (0> ] (19) 


the derivative of the network function, 3g/3x*°\ 
can be calculated recursively as 


3g(k)[t(k)] 
ax ( k-i)= 0 t (k) 


w (k-l) 


(20) 


Since the weight matrices are constant, the differen- 
tiability of the network is determined by the activa- 


tion function used in each node. One of the most 
common activation functions is the sigmoid 

° (z) = TmT^ Vz€R (21) 

Since this function is infinitely differentiable, the 
full network also is infinitely differentiable. 

The basic CMAC network is very different from the 
feedforward network. The quantization of the in- 
put space causes the CMAC to have piecewise-con- 
tinuous output as demonstrated in Fig. 5 for a 
coarse quantization of the input value. 



Figure 5. Example CMAC output for very coarse 
receptive regions. 

Although useful in many other situations, the basic 
CMAC cannot be used in NID control because the 
control laws require continuous derivatives. 
Noticing the similarity between the receptive re- 
gions of CMACs and first-order basis splines. Lane 
et al. introduce highcr-ordcr CMACs in [26]. Equation 
15 is redefined to be 

a .J B n.j< x > xeA i (22 

1 l 0 otherwise ^ 

where the basis functions, B, of order, n, differ from 
conventional B-splines [basis functions] [1,2, 26] to 
account for differing support (or receptive) regions. 
Due to the similarity of the basis functions, higher- 
order CMACs inherit the continuity properties of B- 
splines, including (n - 2) continuous derivatives. 
The penalty paid for this increase in continuity is 
the increased computation necessary in eq. 22 when 
compared to eq. 15. An example of receptive re- 
gions for a higher-order CMAC is given in Fig. 6. 



Figure 6. Receptive Regions of a Higher-Order 
CMAC. 
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EXAMPLE NEURAL NETWORK SYSTEM 
IDENTIFICATION 

As a preliminary step to the full implementation of 
the integrated neural-network, NID control law, a 
simple investigation of the capabilities of the neu- 
ral networks was performed. A one-dimensional 
test function representing lift coefficient variation 
with angle-of-attack for the airplane example is 
given in Fig. 7. 



Figure 7. Lift curve test function 

The learning capabilities of neural networks were 
tested by presenting random angles-of-attack be- 
tween -5° and 25°, and the corresponding lift coeffi- 
cients to each network. The result after 50,000 ran- 
dom training samples presented to a back-propaga- 
tion network with two hidden layers of 10 and 5 
nodes respectively, is given in Fig. 8. The error be- 
tween the network output and the actual lift curve 
is very small, indicating a good representation of 
the original function. 



Figure 8. Back-propagation error for lift curve test 
function 

The basic CMAC network network is similarly 
tested by presenting random input/output pairs. 
The result after 2000 such presentations for a net- 
work with 100 weights and a generalization value, 
C, of 10 (Fig. 9) also is very good. The noisy appear- 


ance of the network error is due to the piecewise 
continuity of the CMAC output. Higher-order 
CMACs will alleviate this problem. 



Figure 9. CMAC error for lift curve test function 

Further testing of the neural networks, including 
multi-dimensional inputs and higher-order 
CMACs, remains to be completed. Full implemen- 
tation of the integrated neural-network, NID con- 
trol law (Fig. 2) will follow’. 

CONCLUSIONS 

Neural networks provide a feasible alternative for 
function approximation in adaptive (or learning) 
nonlinear control systems. Parallel neural net- 
works representing generalized forces due to state 
and control can be trained using the outputs of an 
extended Kalman filter. Neural network outputs 
must be continuously differentiable for use with 
nonlinear-inverse-dynamic control logic. Back-pro- 
pagation feedforward neural networks using sig- 
moidal activation functions have the necessary dif- 
ferentiability. Cerebellar model articulation con- 
troller neural networks must be modified to 
achieve the required smoothness. Determining the 
practicality of neural-network/nonlinear-inverse- 
dynamic controllers is an important area for future 
research. 
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